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ABSTRACT 

Sequential  quadratic  programming  (SQP)  methods  for  nonlincarly  constrained  optimization  typ¬ 
ically  use  a  merit  function  to  enforce  convergence  from  an  arbitrary  starting  point.  We  define 
a  smooth  augmented  Lagrangian  merit  function  in  which  the  Lagrange  multiplier  estimate  is 
treated  as  a  separate  variable,  and  inequality  constraints  are  handled  by  means  of  non-negative 
slack  variables  that  are  included  in  the  lincsearch.  Global  convergence  is  proved  for  an  SQP 
algorithm  that  uses  this  merit  function.  Wc  also  prove  that  steps  of  unity  are  accepted  in  a 
neighborhood  of  the  solution  when  this  merit  function  is  used  in  a  suitable  superlincarly  conver¬ 
gent  algorithm.  Finally,  a  selection  of  numerical  results  is  presented  to  illustrate  the  performance 
of  the  associated  SQP  method. 


The  material  contained  in  this  report  is  based  upon  research  supported  by  the  U.S.  Department 
of  Energy  Contract  DE-AA03-76SF0032C,  PA  No.  DE-AS03-76ER72018;  National  Science  Foun¬ 
dation  Grants  DCR-8413211  and  ECS-8312142;  the  Office  of  Naval  Research  Contract  N00014- 
85-K-0343;  and  the  U.S.  Army  Research  Office  Contract  DAAG29-84-K-0156. 
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1.  Sequential  Quadratic  Programming  Methods 

Sequential  quadratic  programming  (SQP)  methods  are  widely  considered  the  most  effective  gen¬ 
eral  techniques  for  solving  optimization  problems  with  uouliuear  constraints.  (For  a  survey  of 
results  and  references  on  SQP  methods,  see  Powell,  1983.)  One  of  the  major  issues  of  interest  in 
recent  research  on  SQP  methods  has  been  the  choice  of  merit  function--  the  measure  of  progress 
at  each  iteration.  This  paper  describes  some  properties  of  a  theoretical  SQP  algorithm  (NPSQP) 
that  uses  a  smooth  augmented  Lagrangian  merit  function.  NPSQP  is  a  simplified  version  of  an 
SQP  algorithm  that  has  been  implemented  as  the  Fortran  code  NPSOL  (Gill  et  a f.,  1985,  1986a, 
b).  (The  main  simplifications  involve  the  form  of  the  problem  and  strengthened  assumptions.) 

For  ease  of  presentation,  we  assume  that  all  the  constraints  are  nonlinear  inequalities.  (The 
theory  applies  in  a  straightforward  fashion  to  equality  constraints.)  The  problem  to  be  solved  is 
thus: 

NP  minimize  F(x) 

*£»• 

subject  to  Ci(x)  >  0,  t  =  l,...,m, 

where  F  and  {c<}  are  twice-continuously  differentiable.  Let  g(x)  denote  the  gradient  of  F(x), 
and  A(x)  denote  the  Jacobian  matrix  of  the  constraint  vector  c(x).  The  solution  of  NP  will  be 
denoted  by  x. 

We  assume  that  the  first-order  Kuhu-Tucker  conditions  hold  (with  strict  complementarity) 
at  x*  i.c.,  that  there  exists  a  Lagrange  multiplier  vector  A*  such  that 


g(x)  =  A(x)T$i 

(1-1«) 

© 

II 

£ 

(1.16) 

A*  >  0  if  Ci(x)  =  0. 

(1.1c) 

(For  a  detailed  discussion  of  optimality  conditions,  sec,  for  example,  Fiacco  and  McCormick, 
1968,  and  Powell,  1974.) 

At  the  fc-th  iteration,  the  new  iterate  ifc+1  is  defined  as 

**  +  i  =  **  +  a»p*,  (1.2) 

where  x*  is  the  current  iterate,  p*  is  an  n-vector  (the  search  direction),  and  a*  is  a  non-negative 
step  length  (0  <  a*  <  1).  For  simplicity  of  notation,  we  henceforth  suppress  the  subscript 
k,  which  will  be  implicit  on  unbarred  quantities.  A  barred  quantity  denotes  one  evaluated  at 
iteration  k  +  1. 

The  central  feature  of  an  SQP  method  is  that  the  search  direction  p  in  (1.2)  is  the  solution 
of  a  quadratic  programming  subproblem  whose  objective  function  approximates  the  Lagrangian 
function  and  whose  constraints  are  linearizations  of  the  nonlinear  constraints.  The  usual  definition 
of  the  QP  subproblem  is  the  following: 

minimize  gTp  +  \pTHp 
pe»"  1 

subject  to  Ap  >  — c, 


(1.3a) 

(136) 
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where  g,  c  and  A  denote  the  relevant  quantities  evaluated  at 
positive-definite  quasi- Newton  approximation  to  the  Hessian 
Lagrange  multiplier  vector  p  of  (1.3)  satisfies 

x.  Thtv  matrix  //  is  a  symmetric 
of  the  Lagrangian  function.  The 

g  +  Hp  =  ATp\ 

(1.4«) 

pT{Ap  +  c)  =  0; 

(1-4*0 

/i  >  0. 

(1.4c) 

The  remainder  of  this  paper  is  organized  as  follows.  Section  2  gives  some  background  on 
the  role  of  merit  functions  within  SQP  methods,  and  introduces  the  augmented  Lagrangian  merit 
function  used  in  NPSQP.  In  Section  3  we  state  the  assumptions  about  the  problem  and  present 
the  algorithm.  Global  convergence  results  are  given  in  Section  4.  Section  5  shows  that  the  chosen 
merit  function  will  not  impede  superlinear  convergence.  In  Section  6  we  present  a  selection  of 
numerical  results  to  indicate  the  robustness  of  the  associated  SQP  method  NPSOL. 

2.  Background  on  Merit  Functions 

2.1.  Introduction 

Many  authors  have  studied  the  choice  of  steplcngth  in  (1.2).  Usually,  a  is  chosen  by  a  lincsearch 
procedure  to  ensure  a  “sufficient  decrease”  (Ortega  and  Rhcinboldt,  1970)  in  a  merit  function 
that  combines  the  objective  and  constraint  functions  in  some  way.  A  popular  merit  function  for 
several  years  (Han,  1976,  1977;  Powell,  1977)  has  been  the  l\  penalty  function  (Pietrzykowski, 
1969): 

P{x,p)  =  F(z)  +  p'jT  max(0, -Ci(i)).  (2.1) 

This  merit  function  has  the  property  that,  for  p  sufficiently  large,  x*  is  an  unconstrained  minimum 
of  P(x,p).  In  addition,  p  can  always  be  chosen  so  that  the  SQP  search  direction  p  is  a  descent 
direction  for  P(x,p).  However,  Maratos  (1978)  observed  that  requiring  a  decrease  in  P(x,p)  at 
every  iteration  could  lead  to  the  inhibition  of  superlinear  convergence  (see  Chamberlain  et  a!., 
1982,  for  a  procedure  designed  to  avoid  this  difficulty).  Furthermore,  P(x,p)  is  not  differentiable 
at  the  solution,  and  lincsearch  techniques  based  on  smooth  polynomial  interpolation  are  thus  not 
applicable. 

An  alternative  merit  function  that  has  recently  received  attention  is  the  augmented  La¬ 
grangian  function ,  whose  development  we  now  review.  If  all  the  constraints  of  NP  are  equalities, 
the  associated  augmented  Lagrangian  function  is: 

L(x,X,p)  =  F(x)  -  \Tc(x)  +  \pc(x)Tc(x),  (2.2) 

where  A  is  a  multiplier  estimate  and  p  is  a  non-negative  penalty  parameter.  Augmented  La¬ 
grangian  functions  were  first  introduced  by  Hestcnes  (1969)  and  Powell  (1969)  as  a  means  of 
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creating  a  sequence  of  unconstrained  subproblems  for  the  equality-constraint  ease.  A  crucial 
property  of  (2.2)  is  that  there  exists  a  finite  p  such  that  for  all  p  >  p,  x  is  an  unconstrained 
minimum  of  (2.2)  when  A  =  A*  (see,  e.g.,  Fletcher,  1974).  The  use  of  (2.2)  as  a  merit  function 
within  an  SQP  method  was  suggested  by  Wright  (1976)  and  Schittkowski  (1981). 

2.2.  The  Lagrange  multiplier  estimate 

When  (2.2)  is  used  as  a  merit  function,  it  is  not  obvious — even  in  the  equality-constraint  case 
how  the  multiplier  estimate  A  should  be  defined  at  each  iteration. 

Most  SQP  methods  (e.g.,  Hun,  1976;  Powell,  1977)  define  the  approximate  Hessian  of  the 
Lagrangian  function  using  the  QP  multiplier  p  (c f.  (1.4)),  which  can  be  interpreted  as  the  “latest” 
(and  presumably  “best”)  multiplier  estimate,  and  requires  no  additional  computation.  However, 
using  p  as  the  multiplier  estimate  in  (2.2)  has  the  effect  of  redefining  the  merit  function  at 
every  iteration.  Thus,  since  there  is  no  monotonicity  property  with  respect  to  a  single  function, 
difficulties  may  arise  in  proving  global  convergence  of  the  algorithm. 

Powell  and  Yuan  (1984)  have  recently  studied  an  augmented  Lagrangian  merit  function  for 
the  equality-constraint  case  in  which  A  in  (2.2)  is  defined  as  the  least-squares  multiplier  estimate, 
and  hence  is  treated  as  a  function  of  x  rather  than  as  a  separate  variable.  (An  augmented 
Lagrangian  function  of  this  type  was  first  introduced  and  analyzed  as  an  exact  penalty  function 
by  Fletcher,  1970.)  Powell  and  Yuan  (1984)  prove  several  global  and  local  convergence  properties 
for  this  merit  function. 

Other  smooth  merit  functions  have  been  considered  by  Dixon  (1979),  DiPillo  and  Grippo 
(1979),  Bartholomew-Biggs  (1983,  1985),  and  Boggs  and  Tolle  (1984,  1985)  (the  latter  only  for 
the  equality-constraint  case). 

An  approach  that  makes  an  alternative  use  of  the  QP  multiplier  estimate  is  to  treat  the 
elements  of  A  as  additional  variables  (rather  than  to  reset  A  at  every  iteration).  Thus,  a  “search 
direction”  (  is  defined  for  the  multiplier  estimate,  and  the  linescarch  is  performed  with  respect 
to  both  x  and  A.  This  idea  was  suggested  by  Tapia  (1977)  in  the  context  of  an  unconstrained 
subproblem  and  by  Schittkowski  (1981)  within  an  SQP  method.  This  approach  will  also  be  taken 
in  NPSQP. 


2.3.  Treatment  of  inequality  constraints 

When  defining  a  merit  function  for  a  problem  with  inequality  constraints,  it  is  necessary  to  identify 
which  constraints  are  “active”.  The  l\  merit  function  (2.1)  includes  only  the  violated  constraints. 
The  original  formulation  of  an  augmented  Lagrangian  function  for  inequality  constraints  is  due 
to  Boekafellar  (1973): 

L(x,X,p)  =  F(x)  -  A Tc+(z)  +  \pc>{x)Tc>( x),  (2.3) 


(c,)i 


c,  if  pc,  <  Ajj 
A  i/p  otherwise. 


where 
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Some  global  results  are  given  for  this  merit  function  in  Schitt.kowski  (1981,  1983). 

A  disadvantage  of  (2.3)  as  a  merit,  function  is  that  discontinuities  in  the  first,  derivative  may 
cause  difficulties  for  linoseaich  techniques  based  on  polynomial  interpolation.  Therefore,  to  retain 
smoothness  du.ing  the  linesearch  even  with  inequality  constraints,  we  augment  the  variables  (x,  A) 
by  a  set  of  slack  variables  that  are  used  only  in  the  line  search.  At  the  fc-th  major  iteration,  a 
vector  triple 


V  =  £ 


is  computed  that  serves  as  a  direction  of  search  for  the  variables  (x,A,  s).  The  new  values  are 
defined  by 


=  I  A  I  +a I  ^  I  , 


where  the  vectors  p,  £  and  q  are  found  from  the  QP  subproblem  (1.3),  as  described  below. 

In  our  algorithm  (as  in  Schittkowski,  1981),  £  is  defined  as 

M  -  A,  (2.6) 

so  that  if  a  =  1,  A  =  p.  We  take  A0  as  po  (the  QP  multipliers  at  x0),  so  that  Aj  =  p0  regardless 
of  a0. 

The  definition  of  s  and  q  can  be  interpreted  in  terms  of  an  idea  originally  given  by  Rockafcllar 
(1973)  in  the  derivation  of  (2.3).  In  our  algorithm,  the  augmented  Lagrangian  function  includes 
a  set  of  non-negative  slack  variables: 

L(x,  A,  s,p)  =  F(x)  -  AT(c(x)  -  s) ip(c(x)  -  s)T(c(x)  -  s),  with  s  >  0.  (2.7) 

The  vector  s  at  the  beginning  of  iteration  k  is  taken  as 

fnwxfl),*)  if„  =  0; 

(  max(0,Cj  -  Aj/p)  otherwise. 

(For  given  values  of  x,  A  and  p ,  the  vector  s  defined  by  (2.8)  yields  the  value  of  L  minimized  with 
respect  to  the  slack  variables  alone,  subject  to  the  non-negativity  restriction  s  >  0.)  The  vector 
q  in  (2.5)  is  then  defined  by 


so  that 


Ap  -  q  =  -(c  -  s), 


Ap  -f  c  =  s  -t-  q. 


(2.9  a) 


(2.96) 


We  see  from  (2.9)  that  s  +  q  is  simply  the  residual  of  the  inequality  constraints  from  the  QP  (1.3). 
Therefore,  it  follows  from  (1.3)  and  (1.4)  that 


s  +  q  >  0  and  pT(s  -f  q)  =  0. 


(2.10) 


3.  Statement  uf  Algorithm 
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2.4.  Choice  of  the  penalty  parameter 

Finally,  we  consider  the  definition  of  the  penalty  parameter  in  (2.7).  Our  numerical  experiments 
have  suggested  strongly  that  efficient  performance  is  linked  to  keeping  the  penalty  parameter 
as  small  as  possible,  subject  to  satisfying  the  conditions  needed  for  convergence.  Hence,  our 
strategy  is  to  maintain  a  “current”  p  that  is  increased  only  when  necessary  to  satisfy  a  condition 
that  assures  global  convergence. 

Several  authors  have  stated  that  the  need  to  choose  a  penalty  parameter  adds  an  arbitrary 
element  to  an  SQP  algorithm,  or  leads  to  difficulties  in  implementation.  On  the  contrary,  we 
shall  see  (Lemma  4.3)  that  in  NPSQP  the  penalty  parameter  is  based  directly  on  the  condition 
needed  for  global  convergence,  anil  hence  should  not  be  considered  as  arbitrary  or  heuristic. 

3.  Statement  of  Algorithm 

We  make  the  following  assumptions: 

(i)  x  and  x  +  p  lie  in  a  closed,  bounded  region  fl  of  Rn  for  all  k; 

(ii)  F,  {ci},  and  their  first  and  second  derivatives  are  uniformly  bounded  in  norm  in  0; 

(iii)  H  is  positive  definite,  with  bounded  condition  number,  and  smallest  eigenvalue  uniformly 
bounded  away  from  zero,  i.e.,  there  exists  7  >  0  such  that,  for  all  k, 

pTHp  >  7|!pII2; 

(iv)  ||//||  is  uniformly  bounded  at  every  iterate. 

(v)  The  quadratic  program  (1.3)  always  has  a  solution. 

The  following  notation  will  be  used  in  the  remainder  of  this  section.  Given  x.  A,  s,  p,  (,  q 
and  p,  we  let  4>{n)  denote  L(x  +  ap,  A  +  a(,s  +  aq,p),  i.e.,  the  merit  function  as  a  function  of 

the  steplength.  The  derivative  of  <j>  with  respect  to  a  will  be  denoted  by  <f>'.  Let  v  denote  the 

“extended”  iterate  (1,  A,s),  and  let  y  denote  the  associated  search  direction  (p,(,q).  For  brevity, 
we  sometimes  use  the  notation  “(a)”  to  denote  evaluation  at  v  +  ay. 

The  steps  of  each  iteration  of  algorithm  NPSQP  are: 

1.  Solve  (1.3)  for  p.  If  p  =  0,  set  A  =  p  and  terminate.  Otherwise,  define  £  =  p  —  A. 

2.  Compute  s  from  (2.8).  Find  p  such  that  <p'(0)  <  —  |  pTHp  (see  Lemma  4.3,  below). 

3.  Compute  the  steplength  a,  as  follows.  If 

<fi(l)  -  *(0)  <  <t0'(O)  (3.1a) 

and 

*'(1)  <  V4>'(0)  or  |tf'(l)|  <  -ij^(O),  (3.16) 

where  0  <  a  <  tj  <  A,  set  a  =  1.  Otherwise,  use  safeguarded  cubic  interpolation  (see,  e.g., 

Gill  and  Murray,  1974),  to  find  an  a  6  (0, 1)  such  that 

<f>(a)  -  (f>( 0)  <  <Ja<f>'( 0)  (3.2a) 

and 


-f- 


■  fci'A'A.k.  AT.  in.  ,im 
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l*'(«)l  <  ~V4>'( 0).  (3.2b) 

4.  Update  II  so  that  (iii)  is  satisfied  (e.g.,  using  a  suitably  modified  DFGS  update;  see  Gill  vt 
til.,  198Gb,  for  the  update  used  in  NPSOL). 

5.  Update  x  and  A  using  (2.5). 


4.  Global  Convergence  Results 

In  order  to  prove  global  convergence,  we  first  prove  a  set  of  lemmas  that  establish  various  prop¬ 
erties  of  the  algorithm. 

Lemma  4.1.  Using  Algorithm  NPSQP.  ||p||  =  0  if  and  only  if  x  is  a  Kuhn-Tuckcr  point  of  NP. 
Furthermore,  for  x  sufficiently  close  to  x*  the  active  set  of  the  QP  (1.3)  is  the  same  as  the  active 
set  of  nonlinear  constraints  for  NP. 

Proof.  See  Robinson  (1974).  | 

Lemma  4.2.  For  all  k  >  1, 

llA*>||  <  max  H/iill, 

1<»<  fc-1 

and  hence  ((A/t|(  is  bounded  for  all  k.  In  addition,  ||£*|j  is  uniformly  bounded  for  all  k. 

Proof.  By  definition, 


Ai  —  /*<>; 

Ak+ 1  =  A*  +  Ofc(/*fc  —  Afc),  fc  >  1.  (4.1) 

The  proof  is  by  induction.  The  result  holds  for  Aj  because  of  assumption  (iv)  about  the  bound¬ 
edness  of  ||/ifc||  for  all  k.  Assume  that  the  lemma  holds  for  A*.  From  (4.1)  and  norm  inequalities, 
we  have 

l|AfeM||<afe||/tfe||  +  (l-alt)||A*||. 

Since  0  <  a  <  1,  applying  the  inductive  hypothesis  gives 

l|Afcfi||  <  max  ||pi||,  i  =  l,...,fc, 

which  gives  the  first  desired  result. 

The  boundedness  of  ||£j.||  follows  immediately  from  its  definition  (2.6),  assumption  (iv),  and 
the  first  result  of  this  lemma.  | 

The  next  lemma  establishes  the  existence  of  a  non-negative  penalty  parameter  such  that  the 
projected  gradient  of  the  merit  function  at  each  iterate  satisfies  a  condition  associated  with  global 
convergence. 


4-  (llohal  Convergence  Results 
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Lemma  4.3.  There  exists  p  >  0  such  that 


</>'((),/))  <  -\pTHp 


for  all  p>  p. 


Proof.  The  gradient  of  L  with  respect  to  x,  A  and  s  is  given  by 

/  ff(x)  -  A(x)TX  +  pA{x)T{c(x)  -  s)  \ 

V£(z,  A,  s)  =  I  -(c(x)-»)  ,  (4.3) 

V  *  -  p(c(x)  -  s)  J 

and  it  follows  that  r/>'(0)  is  given  by 

=  PT 9  -  pT AT A  +  ppTAT(c  -  s)  -  (c  -  s)T£  +  A Tq  -  pqT(c  -  s),  (4.4) 

where  g,  A ,  and  c  are  evaluated  at  x. 

Multiplying  (1.4a)  by  pT  gives 

gTp  =  pTATp  -  pTHp.  (4.5) 

Substituting  (2.6),  (2.9a)  and  (4.5)  in  (4.4),  we  obtain 

tf>'(0)  =  ~pr Hp  +  qTp  -  2(c  -  s)Tt  -  p(c  -  s)T(c  -  s).  (4.6) 

Substituting  (4.6)  in  the  desired  inequality  (4.2)  and  re-arranging,  we  obtain  as  the  condition  to 
be  satisfied 

qTp  ~  2 (c  -  s)t£  -  p(c  -  s)T(c  -  s)  <  \pTHp.  (4.7) 

The  complementarity  conditions  (1.4)  and  definition  (2.9)  imply  that  qTp  <  0.  Hence,  if 
\pTHp  >  — 2(c  —  s)t£,  then  (4.7)  holds  for  all  non-negative  p,  and  p  may  be  taken  as  zero.  (Note 
that  this  applies  when  c  -  s  is  zero.)  The  determination  of  p  is  non-trivial  only  if 

\pTHp  <  -2(c  -  s)rjf.  (4.8) 

Re-arranging  (4.7),  we  see  that  p  must  satisfy 

p(c  -  s)T(c  -  s)  >  qTp  -  \pTHp  -  2(c  -  s)t£. 

A  value  p  such  that  (4.8)  holds  for  all  p  >  p  is  given  by 

m 


The  penalty  parameter  p  in  NPSQP  is  computed  by  maintaining  a  “current”  value  of  p  and 
modifying  it  if  necessary  to  satisfy  (4.2).  Thus,  the  “new”  penalty  parameter  p  is  defined  by 

f  p  if  (4.2)  holds; 

p={  n  s  (4.io) 

1  inax(p,  2p)  otherwise, 

where  p  is  defined  by  Lemma  4.3.  Note  that  p  can  become  unbounded  only  if  p  is  unbounded. 
Furthermore,  (4.10)  implies  that  if  p  is  bounded  over  an  infinite  sequence  of  iterations,  it  must 
eventually  reach  a  finite  value,  and  will  retain  that  value  for  all  subequent  iterations. 

VVe  now  show  that,  even  if  the  penalty  parameter  is  unbounded,  pj|pj|2  remains  bounded. 
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Lemma  4.4.  If  (4.8)  holds,  then 

pIIpI!2  <  (4.ii) 

where  7  is  a  lower  bound  on  the  smallest  eigenvalue  .if  H  (cf.  assumption  (in)). 

Proof.  From  (4.8)  and  norm  inequalities,  we  have 

lpTffp  <  — 2(c  —  s)t£  <  2\\c  -  s||||£||,  (4.12) 

which  implies 

ik-s||>^|p.  (4.13) 

Combining  (4.9)  and  (4.13)  and  noting  that  pTHp  >  7||p||2,  we  obtain 

. .  m\2  ^  8ii*ii2 

\pTHp  7||p||2 ' 

The  desired  result  follows  immediately.  | 

The  next  two  lemmas  establish  the  existence  of  a  step  bounded  away  from  zero,  independent 
of  k  and  the  size  of  p,  for  which  a  sufficient  decrease  condition  is  satisfied. 

Lemma  4.5.  For  0  <  a  <  1, 

4>"(a)  =  -</>'(0)  +  qTp  +  iV||p||2, 
where  |jV|  is  bounded  and  independent  ofk. 


Proof.  Using  (4.3),  we  have 


/  V2F  -  +  P('i  ~  ^))V2Ci  +  pATA 

-At 

-pA 

V2L=  -A 

0 

I 

V  ~pA 

I 

Pi 

so  that 

4>”(a)  -  y tV2L(v  +  ay)y  =  pTW(a)p  -  p  (c<(a)  -  si(rt))pTV2c<(a)p 

+  P{M<*)P  -  ?)T(^(«)P  ~  <?)  ~  2(T(A(a )p  -  q), 

where 

W(a)  =  V2F(a)  -  £(*<  +  a*,)V2c,(a). 


(4.14) 


We  now  derive  bounds  on  the  first  two  terms  on  the  right-hand  side  of  (4.14).  The  first 
term  is  bounded  in  magnitude  by  a  constant  multiple  of  ||p]|2  because  of  assumption  (ii)  and  the 
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boundedness  of  ||A||  (from  Lemma  4.2).  For  the  second  term,  we  expand  in  a  Taylor  series 
about  x : 

Ci(x  +  ap)  =  Ci(x)  +  aa^(x)Tp  +  |a2pTV2Cj(x  +  9{p)p ,  (4.15) 

where  0  <  9i  <  a.  Since  s*(a)  =  Sj  +  aq j,  using  (2.9n)  and  multiplying  by  p,  we  have 

p  (c, ( x  +  atp)  -  ( s ,  +  on;,))  =  p(l  -  a)(c,(z)  -  s.)  +  p \a2pTV2a(x  +  9{p)p.  (4.16) 

The  term  |p(c^(x)  -  s,)|  is  uniformly  bounded;  this  follows  trivially  from  the  boundedness  of  |r,| 
and  |.s-,|  if  p  is  bounded,  and  from  (4.9)  if  p  is  unbounded.  Because  ||V2Ci||  is  uniformly  bounded 
in  the  region  of  interest,  a  similar  argument  (using  Lemma  4.4  if  p  is  unbounded)  shows  that 
|ppTV2Cj(x  4-  9,p)p\  is  uniformly  bounded.  Therefore, 

|p(ci(a)  -  Sj(a))|  <  Ji ,  (4.17) 

where  J,  is  bounded  and  independent  of  the  iteration.  Using  (4.17),  we  obtain  the  overall  bound 

1 p  Ma)  “  si(°‘))pTV2ci(i*)p\  <  <7|bl|2.  (4-18) 

where  ,/  is  bounded  and  independent  of  the  iteration. 

Now  we  examine  the  third  term  on  the  right-hand  side  of  (4.14).  Using  Taylor  series,  we 

have 

a{(x  +  ap)Tp  -  ajp  4-  apTV2Ci{9i)p,  (4.19) 

where  0  <  0,  <  a.  From  (2.96)  and  the  boundedness  of  ||p(c  —  s)||  (and  Lemma  4.4  if  p  is 
unbounded),  we  obtain 

ppT  (/1(a)  -  q)  T(/l(a)p  -  ?)  =  p(c  -  s)T(c  -  s)  +  L\\p\\2 ,  (4.20) 

where  \L\  is  bounded  and  independent  of  the  iteration. 

Using  (4.19)  and  the  boundedness  of  ||£]|,  the  final  term  on  the  right-hand  side  of  (4.14)  can 
In-  written  as 

-2 (T(A(a)p  -  q)  =  2£T(c(x)  -  s)  +  Af||p||2,  (4.21) 

where  1  A/ i  is  bounded  and  independent  of  the  iteration. 

Combining  ( 1.20)  and  (4.21),  the  last  two  terms  on  the  right-hand  side  of  (4.14)  become 

p  (A(a)p  -  q)T(A(ot)p  -  q)  -  2(T(A(a)p  -  q)  =  p(c  -  s)T(c  -  s)  +  2 (T(c  -  s)  +  M||p||2 

=  -<£f(0)  4-  qT fi  +  M||p|)2, 

where  M j  is  bounded  and  independent  of  the  iteration  (using  (4.6)  and  noting  that  the  largest 
eigenvalue  of  If  is  bounded). 

Combining  all  these  bounds  gives  the  required  result.  | 
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Lemma  4.6.  The  linesearch  in  Step  3  of  the  algorithm  defines  a  step  length  a  (0  <  a  <  1)  such 
that 

c/>(a)  -  0(0)  <  <ra0'(O),  (4.22) 

and  a  >  d,  where  0  <  a  <  1,  and  d  >  0  is  bounded  away  from  zero  and  independent  of  the 
iteration. 

Proof.  If  both  conditions  (3.1)  are  satisfied,  then  a  =  1  and  (4.22)  holds  with  a  trivially  bounded 
away  from  zero. 

Assume  that  (3.1)  does  not  hold  (i.e.,  a  is  computed  by  safeguarded  cubic  interpolation).  The 
existence  of  a  step  length  ci  that  satisfies  conditions  (3.2)  is  guaranteed  from  standard  analysis 
(see,  for  example.  More  and  Sorensen,  1984).  We  need  to  show  that  a  is  uniformly  bounded  away 
from  zero.  There  are  two  cases  to  consider. 

First,  assume  that  (3.1a)  does  not  hold,  i.e.,  0(  1)  —  0(0)  >  cr0'( 0).  Since  0'(O)  <  0,  this 
implies  the  existence  of  at  least  one  positive  zero  of  the  function 

0(a)  =  0(a)  -  0(0)  -  <ra0'(O). 

Let  a  denote  the  smallest  such  zero.  Since  0  vanishes  at  zero  and  a  ,  and  0'(O)  <  0,  the  mean- 
value  theorem  implies  the  existence  of  a  point  a  (0  <  a  <  a* )  such  that  0'(d)  =  0,  i.e.,  for 
which 

0'(a)  =  o-0'(  0). 

Because  a  <  ij,  it  follows  that 


0'(d)  -  t?0'( 0)  =  (tr  -  v)0'(O)  >  0. 

Therefore,  since  the  function  0'(a )  —  rj0(O)  is  negative  at  a  =  0,  and  non-negative  at  d,  the 
mean-value  theorem  again  implies  the  existence  of  a  smallest  value  d  (0  <  d  <  d)  such  that 

0'(d)  =  »70,(O).  (4.23) 

The  point  a  is  the  required  lower  bound  on  the  step  length  because  (4.23)  implies  that  (3.2b)  will 
not  be  satisfied  for  any  a  €  [0,  d). 

Expanding  0'  in  a  Taylor  series  gives 

0'(a)  =  0'(O)  +  atf>"(9), 


where  0  <  9  <  a.  Therefore,  using  (4.23)  and  noting  that  r\  <  1  and  0*(O)  <  0,  we  obtain 


0(a)- 4' (0) 

r(9) 


*  ^  0"(0)  ' 


(4.24) 


4 ■  Global  Convergence  Results 


11 


(Since  d  >  0.  9  must  he  such  that  >  0.)  We  seek  a  lower  bound  on  d,  and  hence  an  upper 

bound  on  the  denominator  of  (4.24).  We  know  from  Leinma  4.5  that 

0"(6»)  =  -^(O)-f9T/1  +  iV||p||J, 

and  from  (2.10)  that  qTp  <  0.  Therefore, 


*"(#)<  l*'(0)|  +  |W|||p||2, 


and  hence 

-  >  (i  - 
-  i<mi  +  wiipir 

Dividing  by  |</>'(0)|  gives 

(!-»/) 

"  .  .  WIIpII2  ‘ 

l*'(0)| 


(4.25) 


Since  the  algorithm  guarantees  that  <£'(0)  <  -  j pTHp ,  it  follows  that 

|0'(O)I  >  \pTHp  >  |7l|p||*,  (4-26) 

where  7  is  bounded  below.  Thus,  the  denominator  of  (4.25)  may  be  bounded  above  as  follows: 

, ,  wiwr . , ,  mm1  , ,  ziwi 
l«o)l  -  hNI’  1  ' 

A  uniform  lower  bound  on  a  is  accordingly  given  by 

»  >  (4.27) 

—  7  +  2|iV)  v  ' 


In  the  second  case,  we  assume  that  (3.1a)  is  satisfied,  but  (3.16)  is  not.  In  this  case,  it  must 
hold  that  <^>'(1)  >  0,  and  hence  <£'{a)  must  have  at  least  one  zero  in  (0,1).  If  a  denotes  the 
smallest  of  these  zeros,  d  satisfies  (4.23),  and  (4.27)  is  again  a  uniform  lower  bound  on  the  step 
length.  | 

The  proof  of  a  global  convergence  theorem  is  now  straightforward. 

Theorem  4.1.  Under  assumptions  (i)-(v),  the  algorithm  defined  by  (1.2),  (1.3),  (4.2),  and  (4.22) 
has  the  property  that 

lint  IIP* II  =  0.  (4.28) 

k-*oo 

Proof.  If  ||pfc||  =  0  for  any  finite  k,  the  algorithm  terminates  and  the  theorem  is  true.  Hence  we 
assume  that  ||pfc||  ^  0  for  any  k.  There  are  two  cases  to  consider. 


■t 


■i  wywum  jm  y  *  hj*  ^  i 


*7" V  '  A ■-  T  ".■-'  ..■J '.  v 


l\-. 

>ii 
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First,  assume  that  p  is  unbounded.  In  this  case,  the  discussion  following  Lemma  4.3  shows 
that  p  must  he  unbounded,  and  hence  we  know  from  Lemma  4.4  that 


Mi2  < 


81I4H 

7  P 


(4.29) 


Since  ||£||  is  bounded,  the  desired  result  follows  immediately. 

Second,  assume  that  p  is  bounded.  The  discussion  following  Lemma  4.3  shows  that  in  this 
case  p  must  eventually  become  fixed  at  a  given  finite  value,  i.e.,  there  exists  a  value  p  and  an 
iteration  index  K  such  that  p  =  p  for  all  k  >  K.  We  consider  henceforth  only  such  values  of  k. 

The  proof  is  by  contradiction.  We  assume  that  there  exists  c  >  0  and  K  >  K  such  that 
llPfcll  >  c  for  k  >  K.  Every  subsequent  iteration  must  therefore  yield  a  strict  decrease  in  the 
merit  function  (2.7)  with  p  =  p,  because,  using  (4.22), 


<fi(a)  -  <J>( 0)  <  T)a<f>'(0)  <  ~f/d 7c2  <  0. 

The  two  final  inequalities  are  derived  from  Lemma  4.6,  since  a  >  d,  which  is  uniformly  bounded 
away  from  zero.  The  adjustment  of  the  slack  variables  s  in  Step  2  of  the  algorithm  can  only 
lead  to  a  further  reduction  in  the  merit  function.  Therefore,  since  the  merit  function  with  p  =  p 
decreases  by  at  least  a  fixed  quantity  at  every  iteration,  it  must  be  unbounded  below.  But  this 
is  impossible,  from  assumptions  (i)-(ii)  and  Lemma  4.2.  Therefore,  (4.28)  must  hold.  | 

Corollary  4.1. 

liin  ||xfc  -  i*||  =  0. 


Proof.  The  result  follows  immediately  from  Theorem  4.1  and  Lemma  4.1.  | 

The  second  theorem  shows  that  Algorithm  NPSQP  also  exhibits  convergence  of  the  multiplier 
estimate  A. 

Theorem  4.2.  If  A(x*),  the  Jacobian  matrix  of  the  constraints  active  at  x*  has  full  rank,  then 

lim  || Afc  -  A*||  =  0. 

fc— *oo 

Proof.  If  ||pfc||  —  0,  then  p*  =  A*  (using  (1.1)  and  (1.4));  in  Step  1  of  Algorithm  NPSQP,  A*+i 
is  set  to  A*  and  the  algorithm  terminates.  Thus,  the  theorem  is  true  if  pk  =  0  for  any  k.  We 
therefore  assume  that  ||pfc]|  £  0  for  any  k. 

The  definition  (4.1)  gives 

k 

Afc+i  =  Yllikfli’  (4-3°) 

»=o 

where 

k 

Ikk  =  a'k  and  yik  =  a'  (1  -  or' ),  i  <  k,  (4.31) 
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with  a[,  =  1  and  a'  =  n^,  i  >  1.  (This  convention  is  used  because  of  the  special  initial  condition 
that  A0  ~  no-)  From  Lemma  4.6  and  (4.31),  we  observe  that 


0  <  «  <  aj  <  1  for  all », 

0 


and 


7 ik  <  (1  -  «)  \  «  <  *• 


(4.32a) 

(4.32ft) 

(4.32c) 


Since  we  know  from  Theorem  4.1  that  x*  — >  x*  the  iterates  will  eventually  reach  a  neighbor¬ 
hood  of  x  in  which  the  QP  subproblem  identifies  the  correct  active  set  (Lemma  4.1)  and  i4(x*) 
has  full  rank.  Assume  that  these  properties  hold  for  k  >  K i.  From  the  definition  (1.4)  of  /*, 
assumptions  (ii)-(iii),  and  the  existence  of  a  lower  bound  on  the  smallest  singular  value  of  A,  we 
have  for  k  >  Kx  that  there  exists  a  bounded  scalar  M  such  that 


Pt  =  A  +  Mfcdfctfc, 


(4.33) 


with  |Af*|  <  M,  dk  —  max(||pfc||,  ||x*  -  xfc||)  and  ||*k||  =  1.  Given  any  e  >  0,  Theorem  4.1  and 
Corollary  4.1  also  imply  that  K\  can  be  chosen  so  that,  for  k  >  K\, 


|M*d*|  <  |e. 

We  can  also  define  an  iteration  index  K}  with  the  following  property: 

(1  _  a)h  <  - - - 5— 

~  2(fc  +  l)(l+jH-||A*||) 


(4.34) 


(4.35) 


for  k  >  K 2  +  1,  where  p.  is  an  upper  bound  on  ||p||  for  all  k.  Let  K  =  K?).  Then,  from 

(4.30)  and  (4.33),  we  have  for  k  >  2 K, 

K  h 

A*+1  =  E7**^  +  E  7ik(**  +  Miditi ). 

t  =  0  t-/C  +  l 

Hence  it  follows  from  (4.32ft)  that: 

K  h 

A*  +  l  “  A*  =  ^7 ikilii  ~  X)  +  E 

«=0  »=Jf  +  l 


From  the  bounds  on  ||/ij||  and  ||tj||  we  then  obtain 


K  k 

Pft+i  -  A* II  <  OH-  II^H)X37«  +  E 

»=#f+t 


(4.36) 


•=o 


:'‘:S 
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Since  k  >  2 A,  it  follows  from  (1. 32a)  ami  ( 4.32c)  that 


X lik  <  X(1  -  «)k  '<  V(1  -  n)2K  '  <  (K  +  1)(1  -  ii)1 


»  0  i  U 


Using  (4.35),  we  thus  obtain  the  following  bound  for  the  first  term  on  the  right-hand  side  of 
(4.36): 


(/"*  +  P*!!)  X7,fc  ^  2e- 

i  -0 

To  bound  the  second  term  in  (4.36),  we  use  (-4.326)  and  (4.34): 

k  k 

X  nkWnui  <  X Ti4  -  \e- 


(4.37) 


(4.38) 


Combining  (4.36)- (4.38),  we  obtain  the  following  result:  given  any  e  >  0,  we  can  find  K  such 


which  implies  that 


liAfc  — A||<e  for  k  >  2K  +  1, 


lim  ||  A*  -  A  ||  =  0. 

*--«o c 


5.  Use  within  a  Superlinearly  Convergent  Algorithm 

As  mentioned  in  Section  2.1,  a  point  of  interest  is  whether  superlinear  convergence  may  be 
impeded  by  the  requirement  of  a  sufficient  decrease  in  the  merit  function  at  every  iteration.  In 
this  section  we  show  that  a  unit  step  (a  =  1)  will  satisfy  conditions  (3.1)  when  the  iterates  are 
sufficiently  close  to  the  solution  and  x  and  A  are  converging  superlinearly  at  the  same  rate.  For 
further  discussion  of  superlinear  convergence,  see  Dennis  and  More  (1977)  and  Boggs,  Tolle  and 
Wang  (1982). 

In  addition  to  the  conditions  assumed  in  the  previous  section,  we  assume  that  for  all  suffi¬ 
ciently  large  k: 


xk  +  Pk  -  x  =  °( II x*  -  x*ll)i 
A*  +  (k  -  A*  =  o(||Afc  -  A*  ||), 


>  M  >  0, 


(5.1a) 

(5.16) 

(5.1c) 


where  M  is  independent  of  K.  Note  that  (5.1)  implies 

IIP* I!  ~  ||x*  -  x*||  and  ||£*||  ~  ||A*  -  A*||,  (5.2) 

where  means  that  the  two  quantities  are  of  similar  order.  (See  also  Dennis  and  More,  1977.) 

First  wc  show  that  these  assumptions  imply  that  the  penalty  parameter  p  in  the  merit 
function  (2.7)  must  remain  bounded  for  all  k. 
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Lcmtna  5.1.  Under  assumptions  (i)  (v)  of  Section  3.  and  conditions  (5.1),  there  exists  a  finite 
p  such  that  for  all  fc, 

P  <  P •  (5  3) 


Proof.  Recall  from  the  proof  of  Lemma  4.3  that.  p  can  become  unbounded  only  if  (4.8)  holds 
and  pas  defined  by  (  1.9)  becomes  unbounded.  Condition  (4.8)  states  that  —  2(c  —  s)T{  >  ^pTHp. 
Using  assumption  (iii)  and  norm  inequalities,  (4.8)  thus  implies 


Hence,  from  (5.1c),  we  obtain 


llc  -  sll  > 


hlkil2 
11(11  ' 


Ik  -  *11 >  blbll2  _  hl, 
11(11  “  lidl2 


=  M  >  0. 


Consequently,  from  Lemma  4.4, 


P  < 


Af  ’ 


and  remains  bounded.  This  proves  the  lemma.  | 

The  next  two  lemmas  show  that  conditions  (3.1)  are  eventually  satisfied  for  all  k  sufficiently 
large. 


Lemma  5.2.  Under  assumptions  (i)~(v)  of  Section  3  and  conditions  (5.1),  the  sufficient  decrease 
condition  (3.1a)  holds  for  sufficiently  large  k,  i.e., 


where  0  <  cr  <  i. 


*0)-*(0)<a*'(0), 


Proof.  As  in  Powell  and  Yuan  (1984),  observe  that  the  continuity  of  second  derivatives  gives  the 
following  relationships: 

F(x  +  p)  =  F(x)  +  l  (g(x)  f  g( x  +  p))Tp  +  o(||pj|J), 
c(x  +  p)  =  c(x)  +  (A(x)  +  A(x  +  p))p  +  o(||p||2). 

Conditions  (5.1)  and  (5.2)  then  imply: 

F(x  +  p)  =  F(x)  +  |(<?(r)  +  g(x  ))Tp  +  °(||p||s)>  (5.4o) 

c(x  +  p)  =  c(x)  +  -(^(x)  +  A(x*))p  +  o(||p||J).  (5.46) 

We  shall  henceforth  use  g  to  denote  g(x)  and  g  to  denote  g(x),  and  similarly  for  F,  c  and  A. 
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By  definition. 


,/.(())  -  /•- 

A  r!e 

s,  I  1 

\  '1''  'M'lc  s), 

(5.5a) 

■Ad)  U<r 

■  /'! 

r 

t>  " 

l.r  •  ;»)  -  r  Ap) 

- 

:  p 

)  r  Ap)  '  (e[x  r  p)  ~  c  -  .4p). 

(5.56) 

Using  Taylor  series,  we  have 

<•(  r 

p) 

'  (•  t  .4 /:  1  (  M'l/'ij2). 

(5.6) 

Substituting  from  (5.1)  and 

( :>.(•,) 

into  1 

1 5.5 1>),  we  obtain 

<A(  l)  --  /' 

•  jo 

7  !  !!  ) 

'p  •  ■/<r(-4  ,f)j,  f  o(||p|!2). 

(5.7) 

Combining  (5.5a)  and  (5.7)  give 

s 

<A(  1)  -  <A(0)  --  \pr<)  ‘  Ip1//  •  l/iWp  };/iTA*p  +  \T(c  -  s) 

-  2  -  s)  <>(jjp||2).  (5.8) 

Using  (2.G),  (2.9)  and  (4.1),  we  ol>tain  the  following  expression: 

-  p7y  -1-  2A r(c  -  s)  p{c  -  s)  r(<:  s)  4-  pTAp  --  pTq.  (5.9) 

Substituting  (5.9)  into  (5.8),  wo  have 

<{>(  1)  -  «/>(!))  -  ’•</>'(())  *  \p‘q  ■  '2 /»rl v*  •  .4*  /t)  4-  o(|jp||2).  (5.10) 

p 

It  follows  from  (5.15)  (5.1c)  that  the  expression  y  1  //  i-.  o(!lj»2),  which  gives 

vd  1 1  ..MU)  -o'nO  •  \,,rq  ■  o(|!p||2) 

•  }  ■/•'(<•)  >,[  >  2i. 

and  hence 

4>(  1)-<H0)  «  ,;'"i  -  t  '  -  ),"<»»)  o(t;p||2).  (5.11) 

Since  a  <  |  and  <//(())  satisfies  (1  2).  {5.1  !  j  ;n,piie,  that  1.5  la  I  holds  for  sufficiently  large  k.  | 

Lemma  5.3.  Under  assumptions  (:)  (e)  and  cuu.i.i une  (3.1).  the  second  lincsearch  condition 
(3.1b)  holds  for  sufficiently  In  rye  k,  i.e.. 

4>  \  1 ;  -  tpp  (l!)  or  i  1  I  >j  g/ ( 0 ) | . 

where  77  <  i . 

Proof.  In  this  proof,  we  use  the  notation  ’/(l)  to  <  1  •  u ■  >t •  ,px  t  p).  and  similarly  for  c  and  A. 
Using  (4.3),  </>'(l)  is  given  by 

<^(1)  ;r  PTfl(l)  /'^'(lip  •  pp'(oii)  r  .1  />) 

(/'  ^ f  (<■■(  1 )  ••  -•>/*)  •  o'p  1)  c- Ap). 


(5.12) 


From  conditions  (5.2)  (5.3),  wo  have 

9(1)  =  //*  -MIMI2),  .1(1)  =  -4*  +  o(||H|2)  and  p  =  A*  +  o(||p||).  (5.13) 

Substituting  from  (5.0)  and  (5.13)  into  (5.12)  then  gives: 

<*>'(!)  -  pV  -  VTA*h  f  qTp  -  (Ac-  -  s)T(c(l)  -  c  -  Ap)  +  o(||p||2).  (5.14) 

Consider  now  the  vectors  (c  —  s),  q  and  p.  From  (2.8)  and  (5.2),  ||c  —  s||  =  0(||p|j).  From 
Lemma  4.1,  we  know  that  the  QP  subproblem  (1.3)  will  eventually  predict  the  correct  active  set, 
and  hence  that  p,  —  0  if  )  >  0.  For  an  active  constraint  Cj,  it  follows  from  (2.8)  that  will 
eventually  be  set  t.o  zero  at  the  beginning  of  every  iteration  if  p  >  0,  and  hence  q ±  must  also  be 
zero.  If  p  —  0  and  c<  is  an  active  constraint,  then  |g<|  =  o(||p||2).  Therefore,  in  either  case  we 
have  that 

\qTp\  =  o(||p||2)  and  |p(c  -  s)T(c(l)  -  c  -  Ap)|  =  o(||p||2).  (5.15) 

j|i 

Recalling  that  ||</  —  A  /i||  =  o(||p||)  and  using  (5.15)  in  (5.14),  we  obtain 

<m  =  «(iipii2)-  (5.i6) 

Since  |</>'(0)|  >  |7||p||2,  (5.16)  irnples  that  (3.15)  will  eventually  be  satisfied  at  every  iteration. 

I 

6.  Numerical  Results 

In  order  to  indicate  the  reliability  and  efficiency  of  a  practical  SQP  algorithm  based  on  the  merit 
function  (2.7),  we  present  a  selection  of  numerical  results  obtained  from  the  Fortran  code  NPSOL 
(Gill  ct  ill.,  1986a,  b).  Table  1  contains  the  results  of  solving  a  subset  of  problems  70-119  from 
Hock  and  Schittkowski  (1981).  (We  have  omitted  problems  that  are  non-differentiable  or  that 
contain  only  linear  constraints.  Since  NPSOL  treats  linear  constraints  separately,  they  do  not 
affect  the  merit  function.) 

The  problems  were  run  on  an  IBM  3081K,  in  double  precision  (i.e.,  machine  precision  eM 
is  approximately  2.22  x  10  16 ).  The  default  parameters  for  NPSOL  were  used  in  all  cases  (for 
details,  see  Gill  et  nl .,  1986a).  In  particular,  the  default  value  for  ftol,  the  final  accuracy  requested 
in  F.  is  5.4  x  10  12 ,  and  ctol,  the  feasibility  tolerance,  is  Analytic  gradients  were  provided 

for  all  functions,  and  a  gradient  linescarch  was  used. 

For  successful  termination  of  NPSOL,  the  iterative  sequence  of  i-values  must  have  converged 
and  the  final  point  must  satisfy  the  first-order  Kuhn-Tucker  conditions  {ct.  (1.1)).  The  sequence 
of  iterates  is  considered  to  have  converged  at  x  if 

«IIp!I  <  Vft°i  (i  +  INI).  (6.1) 

where  p  is  the  search  direction  and  o  the  step  length  from  (1.2).  The  iterate  x  is  considered 
to  satisfy  the  first-order  conditions  for  a  minimum  if  the  following  conditions  hold:  the  inactive 
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constraints  arc  satisfied  to  within  rt.ol,  the  magnitude  of  each  active  constraint  residual  is  less 
than  ctol,  and 

\\Z{-r)Tn(.r)\\  <  yfjlol  (1  4  max(l  -f  \F(x)\,  !|«(i)||)),  (0.2) 

wliere  Z(x)  is  an  orthogonal  l)asis  for  the  null  space  of  the  gradients  of  the  active  constraints. 
(Thus,  Z[x)Tg{x)  is  the  usual  projected  gradient.) 

Table  1  gives  the  following  information:  the  problem  number  in  Hock  and  Schittkowski; 
the  number  of  variables  (n);  the  number  of  simple  bounds  (m„);  the  number  of  general  linear 
constraints  (mt);  the  number  of  nonlinear  constraints  (mA);  the  number  of  iterations;  and  the 
number  of  function  evaluations.  (Each  constraint  is  assumed  to  include  a  lower  and  an  upper 
bound.) 
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NPSOL  terminated  successfully  on  all  tin-  problems  except  the  badly  scaled  problem  85, 
for  which  (G.l)  could  not  be  satisfied.  However,  the  optimality  conditions  were  satisfied  at  the 
final  iterate,  which  gave  an  improved  objective  value  compared  to  that  in  Hock  and  Schittkowski 
(1981). 
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